IN THE CLAIMS ^\ 

1 . (Currently Amended) A three-dimensional position and orientation sensing 
apparatus comprising: 

image input means for inputting an image acquired by an image acquisition 
apparatus and having showing at least three markers having color or geometric characteristics 
as one image , three-dimensional positional information of which the markers with respect to 
an object to be measured being is known in advance; 

region extracting means for extracting a region corresponding to each marker 
en in the image; 

marker identifying means for identifying the individual markers #em based on 
the color or geometric characteristics of the app e aranc e of the markers in the extracted 
regions; and 



dimensional position and orientation of the object to be measured with respect to the image 
acquisition apparatus, by using positions of the identified markers en in the image input to the 
image input means , and the three-dimensional positional information of the markers with 
respect to the object to be measured. 

2. (Currently Amended) A three-dimensional position and orientation sensing 
apparatus comprising: 

image input means for inputting an image acquired by an image acquisition 
apparatus and having showing at least four markers having color or geometric characteristics 
as one image , three-dimensional positional information of which the markers with respect to 
an object to be measured being is known in advance; 



position and orientation calculating means for calculating the three- 
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region extracting means for extracting an area corresponding to each marker or 

in the image; 

marker identifying means for identifying the individual markers #em based on 
the color or geometric characteristics of the appearanc e of the markers in the extracted 
regions; 

marker selecting means for selecting three markers from the markers identified 
by the marker identifying means; 

parameter calculating means for calculating a plurality of sets of parameters for 
calculating the three-dimensional position and orientation of the object to be measured with 
respect to the image acquisition apparatus, by utilizing positions on th e imag e of the three 
markers in the image input to the image input means which are selected by the marker 
selecting means and three-dimensional positional information of each marker with respect to 
the object to be measured; and 



application of the parameters calculated by the parameter calculating means, to markers not 
selected by the marker selecting means. 



according to claim 2, further comprising parameter improving means for improving one set of 
parameters selected by the parameter selecting means, by evaluating the set of parameters 
through application thereto of three-dimensional positional information of each marker with 
respect to the object to be measured. 

4. (Currently Amended) A three-dimensional position and orientation sensing 
apparatus comprising: 



parameter selecting means for selecting one set of parameters by evaluating 



3. (Original) A three-dimensional position and orientation sensing apparatus 
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image input means for inputting an image acx^uired by an image acquisition 
apparatus and having showing at least three markers having color or geometric characteristic 
as one image, three-dimensional positional information of which the markers with respect to 
an object to be measured is being known in advance; 

region extracting means for extracting a region corresponding to each marker 
en in the image; 

marker identifying means for identifying the individual markers ipem based on 
the color or geometric characteristics of th e app e aranc e of the markers in the extracted 
regions; 

distance estimating means for estimating a distance from the image acquisition 
apparatus to each marker based on th e siz e a size of the extracted region in the image 
including the identified marker in th e imag e; and 

position and orientation calculating means for calculating the three- 
dimensional position and orientation of the object with respect to the image acquisition 
apparatus, by using the estimated distance to each marker, a position of the marker en in the 
image and three-dimensional positional information of this marker with respect to the object. 

(Currently Amended) A three-dimensional position and orientation sensing 
apparatus comprising: 

image input means for inputting an image acquired by an image acquisition 
apparatus and having markers, three-dimensional positional information of which with respect 
to an object to be measured is known in advance; 

imag e r e ducing m e ans for r e ducing th e siz e of th e input imag e ; 
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region extracting means for reducing the size of the input image and extracting 
a region corresponding to each marker on in the reduced image; and 

position and orientation calculating means for calculating the three- 
dimensional position and orientation of the object with respect to the image acquisition 
apparatus, by using the extracted position of each marker en in the image and three- 
dimensional positional information of each marker with respect to the object. 



^ jf> (Original) A three-dimensional position and orientation sensing apparatus 
according to claim 4, further comprising: 

feature extracting means for extracting features of the object to be measured, 
from the image; and 

means for calibrating the position and orientation of the calculated object, from 
the position of the extracted features. 



/ /7. (Original) A three-dimensional position and orientation sensing apparatus 

i 

according to claimA, further comprising: 

feature extracting means for extracting features of the object to be measured, 
from the image; and 

means for calibrating the position and orientation of the calculated object, from 
the position of the extracted features in the image, 

8. (Cxirrently Amended) A three-dimensional position and orientation sensing 
method for measuring the position and orientation of an object to be meas\ired with respect to 
an image acquisition apparatus, by analyzing an image acquired by this image acquisition 
apparatus, the method comprising the steps of: 
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inputting an image acquired by an image acquisition apparatus and having 
showing at least three markers having color or geometric characteristics as one image, three- 
dimensional positional information of which the markers with respect to an object to be 
measured is being known in advance; 

extracting a region corresponding to ea^ marker en in the image; 

identifying the individual markers ftem based on the color or geometric 
characteristics of tho app e aranc e of the markers in the extracted regions; and 

calculating the three-dimensional position and orientation of the object to be 
measured with respect to the image acquisition apparatus, by using positions of the identified 
markers in the image input to the image input in the inputting step , and the three-dimensional 
positional information of the markers with respect to the object to be measured. 

9. (Currently Amended) A three-dimensional position and orientation sensing 
method for measuring a position and orientation of an object to be measured with respect to 
an image acquisition apparatus, by analyzing an image acquired by this image acquisition 
apparatus, the method comprising the steps of: 

inputting an image acquired by an image acquisition apparatus and having 
showing at least four markers having color or geometric characteristics as one image , three- 
dimensional positional information of which the markers with respect to an object to be 
measured is being known in advance; 

extracting a region corresponding to each marker ^ in the image; 

identifying the individual markers &em based on the color or geometric 
characteristics of th e app e arance of the markers in the extracted regions; 
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^^^^^^^"^^^ \ selecting means. 



selecting three markers from the markers identified by the marker identifying 

means; 

calculating a plurality of sets of parameters for calculating the three- 
dimensional position and orientation of the object to be measured with respect to the image 
acquisition apparatus, by utilizing positions in th e image of the three markers in the image 
input in the inputting step which are selected by the marker selecting means and three- 
dimensional positional information of each marker with respect to the object; and 

selecting one set of parameters by evaluating application of the parameters 
calculated by the parameter calculating means, to markers not selected by the marker 



10. (Original) A three-dimensional position and orientation sensing method 
according to claim 9, further comprising the step of improving one set of parameters selected 
by the parameter selecting means, by evaluating the set of parameters through application 
thereto of three-dimensional positional information of each marker with respect to the object. 

1 1 . (Currently Amended) A three-dimensional position and orientation 
sensing method for measuring the position and orientation of an object to be measured with 
respect to an image acquisition apparatus, by analyzing an image acquired by this image 
acquisition apparatus, the method comprising the steps of: 

inputting an image acquired by an image acquisition apparatus and having 
showing at least three markers having color or geometric characteristics as one image , three- 
dimensional positional information of which the markers with respect to an object to be 
measured is being known in advance; 

extracting a region corresponding to each marker en in the image; 
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identifying the individual markers ifem based on the color or geometric 
characteristics of th e app e aranc e of the markers in the extracted regions ar e as ; 

estimating #ie a distance from the image acquisition apparatus to each marker 
based on a size of the extracted region of th e id e ntifi e d mark e r in the image including the 
identified marker : and 

calculating the three-dimensional position and orientation of the object to be 
measured with respect to the image acquisition apparatus, by using the estimated distance to 
each marker, a position of the marker en in the image and three-dimensional positional 
information of this marker with respect to the object. 

(Currently Amended) A three-dimensional position and orientation 
^-""^^ sensing method for measuring the position and orientation of an object to be measured with 

respect to an image acquisition apparatus, by analyzing an image acquired by this image 
acquisition apparatus, the method comprising the steps of: 

inputting an image acquired by an image acquisition apparatus and having 
^^2^^ - markers, three-dimensional positional information of which with respect to an object to be 

measured is known in advance; 

reducing th e input imag e ; 

reducing the input image and extracting a region corresponding to each marker 
on the reduced image; and 

calculating the three-dimensional position and orientation of the object to be 
measured with respect to the image acquisition apparatus, by using the extracted position of 
each marker ^ in the image and three-dimensional positional information of each marker 
with respect to the object to be measured. 
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^0-^^^pl3l^ A three-dimensional position and orientation sensing method 

according to claim 11, further comprising the steps of: 

extracting features of the object to be measured, from the image; and 
calibrating the position and orientation of the calculated object, from a position 

of the extracted features in the image. 

14. (Original) A three-dimensional position and orientation sensing method 
according to clain)!i^, further comprising the steps of: 

extracting features of the object to be measured, from the image; and 
calibrating the position and orientation of the calculated object, from positions 



, of the extracted features in the image. 



15. (Currently Amended) An article of manufacture comprising a computer- 
readable recording medium having computer-readable program coding means as a processing 
program recorded for measuring the position and orientation of an object to be measured with 
respect to an image acquisition apparatus, by analyzing by computer an image acquired by 
this image acquisition apparatus, the computer-readable program coding means comprising: 

computer-readable programming means for making an image to be input, the 
image having been acquired by an image acquisition apparatus and having showing at least 
three markers having color or geometric characteristics as one image , three-dimensional 
positional information of which the markers with respect to an object to be measured being is 
known in advance; 

computer-readable progranmiing means for making on ar e a a region 
corresponding to each marker ee in the image to be extracted; 



h\woric\1373\!3378\amend\13378.ain2 



computer-readable programming means for making the individual markers to 
be identified 6=em based on the color or geometric characteristics of th e app e aranc e of the 
markers in the extracted regions; and 

computer-readable programming means for making the three-dimensional 
position and orientation of the object to be measured with respect to the image acquisition 
apparatus to be calculated, by using positions of the identified markers on th e imag e in the 
image input by the computer-readable programming means , and the three-dimensional 
positional information of the markers with respect to the object to be measured. 

16. (Currently Amended) An article of manufacture comprising a computer- 



^,_v \ readable recording medium having computer-readable program coding means as a processmg 
program recorded for measuring the position and orientation of an object to be measured with 
respect to an image acquisition apparatus, by analyzing by computer an image acquired by 
this image acquisition apparatus, the computer-readable program coding means comprising: 
computer-readable programming means for making an image to be input, the 
image having been acquired by an image acquisition apparatus and having showing at least 
four markers having color and geometric characteristics as one image, three-dimensional 
positional information of which the markers with respect to an object to be measxired being is 
known in advance; 

computer-readable programming means for making a region corresponding to 
each marker en in the image to be extracted, making the individual markers to be identified 
ftem based on the color or geometric characteristics of the app e aranc e of the markers in the 
extracted regions, and making three markers to be selected fi-om the identified markers; 
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computer-readable programming means for making a plurality of sets of 
parameters for calculating the three-dimensional position and orientation of the object to be 
measured with respect to the image acquisition apparatus to be calculated, by using positions 
of the selected three markers in the image input by the computer-readable programming 
means on th e imag e, and the three-dimensional positional information of the markers with 
respect to the object to be measxired; and 

computer-readable progranmiing means for making one set of parameters 
selected, by evaluating the non-selected markers through application of the calculated 
parameters thereto. 

17. (Original) An article of manufacture comprising a computer-readable 
C^^^''^^ recording medium storing a three-dimensional position and orientation sensing processing 

program according to claim 16, wherein 

the computer-readable program coding means as a processing program further 

comprises: 

computer-readable programming means for making one set of parameters 
selected by the parameter selecting means to be improved, by evaluating the set of parameters 
through application thereto of three-dimensional positional information of each marker with 
respect to the object. 

18. (Currently Amended) An article of manufacture comprising a computer- 
readable recording medium having computer-readable program coding means as a processing 
program recorded for measuring the position and orientation of an object to be measured with 
respect to an image acquisition apparatus, by analyzing by computer an image acquired by 
this image acquisition apparatus, the computer-readable program coding means comprising: 
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computer-readable programming means for making an image to be input, the 
image having been acquired by an image acquisition apparatus and having showing at least 
three markers having color or geometric characteristics as one image , three-dimensional 
positional information of which the markers with respect to an object to be measured being is 
known in advance; 

computer-readable progranmiing means for making a region corresponding to 
each marker en in the image to be extracted; 

computer-readable progranmiing means for making the individual markers to 
be identified #em based on the color or geometric characteristics of the appearanc e of the 
markers in the extracted regions; 
^^^^ \^ computer-readable programming means for making a distance fi-om the image 



acquisition apparatus to each marker to be estimated based on ^ a size of the identified 
marker in marker on the image; and 

computer-readable progranmiing means for making the three-dimensional 
position and orientation of the object to be measured with respect to the image acquisition 
apparatus to be calculated, by using the estimated distance to each marker, a position of the 
marker en in the image and three-dimensional positional information of this marker with 
respect to the object to be measured. 

(Currently Amended) An article of manufacture comprising a computer- 
readable recording medium having computer-readable program coding means as a processing 
program recorded for measuring the position and orientation of an object to be measured with 
respect to an image acquisition apparatus, by analyzing by computer an image acquired by 
this image acquisition apparatus, the computer-readable program coding means comprising: 
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computer-readable programming means for making an image to be input, the 
image having been acquired by an image acquisition apparatus and having markers, three- 
dimensional positional information of which with respect to an object to be measured is 
known in advance; 

computer r e adabl e programming means for making th e input imag e to b e 

r e duc e d; 

computer-readable programming means for making the input image to be 
reduced and making a region corresponding to each marker to be extracted on the reduced 
image; and 

computer-readable programming means for making the three-dimensional 
position and orientation of the object to be measured with respect to the image acquisition 
apparatus to be calculated, by using the extracted position of each marker en in the image and 
three-dimensional positional information of each marker with respect to the object. 

I'^j^vO^ (Original) An article of manufacture comprising a computer-readable 
recording medivim storing a three-dimensional position and orientation sensing processing 
program according to claim 18, wherein 

the computer-readable program coding means as a processing program further 

comprises: 

computer-readable programming means for making features of the object to be 
measured, to be extracted from the image, and making the position and orientation of the 
calculated object to be calibrated, from a position of the extracted features. 




■1\ 



(Original) An article of manufacture comprising a computer-readable 



recording medium storing a three-dimensional position and orientation sensing processing 
program according to claim^, wherein 

the computer-readable program coding means as a processing program further 

comprises: 

computer-readable programming means for making features of the object to be 
measured, to be extracted from the image, and making the position and orientation of the 
calculated object to be calibrated, from a position of the extracted features in the image. 

[^ims 22-26 (Withdrawn) 

"P^^^^ ^ \^ /2J^2^>2f^(Currently Amended) An apparatus for measuring a relative three- 

dimensional position and orientation of an object with respect to an image acquisition 
apparatus by measuring image positions of a plurality of at least three markers mounted on the 
object or near the object, based on an image ef showing the markers^ acquired by the image 
^^^^ acquisition apparatus, wherein the markers have color or geometric characteristics^d three- 

dimensional positions of the markers are known in advance, and the measuring apparatus 
comprising: 

means for extracting a region corresponding to each of the plurality of markers 
from within the image of the markers; 

means for identifying the individual markers by calculating color or geometric 
characteristics of the region corresponding to each of the markers within the image; and 

means for estimating the relative three-dimensional position and orientation of 
the object with respect to the image acquisition apparatus, based on the three-dimensional 
positions of the markers. 
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i y 2^. (Original) A three-dimensional position and orientation sensing apparatus 
according to claim 2?; wherein coded patterns are disposed within the individual markers. 

yih. (Original) A three-dimensional position and orientation sensing apparatus 
according to claim^lv, wherein 

the means for identifying the individual markers by calculating geometric 
characteristics of the region of each marker within the image measures the size of the region 
of each marker within the image, and 

the means for estimating the relative three-dimensional position and orientation 
of the object with respect to the image acquisition apparatus calculates an initial estimate of 
the distance between the image acquisition apparatus and a marker based on the measured 
\_ value of the size of the marker region, and calculates the relative three-dimensional position 



— and orientation of the object with respect to the image acquisition apparatus based on the 
initial estimate. 




30. Cancelled 



(Original) A three-dimensional position and orientation sensing apparatus 
according to claini^, wherein an estimate value is calculated by utilizing only a marker that 
takes a range which makes it possible to estimate the relative distance. 

y32. (Original) A three-dimensional position and orientation sensing apparatus 
according to claim^, wherein an estimate error variance is calculated for the estimated 
relative distance, and the three-dimensional position and orientation of the object with respect 
to the image acquisition apparatus is estimated by taking both the estimate value and the error 
variance into consideration. 
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^123. (Original) A three-dimensional position and orientation sensing apparatus 
according to claim 29, wherein the markers are circular markers, a projected image of each 
marker within the image is approximated to an ellipse, and the distance from the image 
acquisition apparatus to each marker is estimated by utilizing the length of the long axis of the 
ellipse. 

^0 




f. (Original) A three-dimensional position and orientation sensing apparatus 
according to claim^, wherein a circular marker is utilized as each marker. 

>35. (Original) A three-dimensional position and orientation sensmg apparatus 
according to claini^, wherein a circular marker is utilized as each marker, and small pattems 
V X different colors are used as a code within the marker. 




(Original) A three-dimensional position and orientation sensing apparatus 
according to claiin28, wherein pattems of different colors are generated on a concentric circle 
as the maker. 

yr (Original) A three-dimensional position and orientation sensmg apparatus 
according to claim 33, wherein a three-dimensional positional sensing is carried out by adding 
direction information of the circular marker. 

3«. (Original)A three-dimensional position and orientation sensing apparatus 
according to claim^, wherein, for extracting a marker region from an original image, the 
apparatus generates a reduced image of the original image, extracts a candidate region that is 
considered to correspond to a marker from the reduced image, calculates the area of the 
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candidate region within the original image, and then extracts and recognizes a region 
corresponding to the marker within the original image. 

19: (Currently Amended) A three-dimensional position and orientation 
sensing apparatus according to claim wherein, after a marker having unique geometric 
characteristics has been recognized, the apparatus extracts from within the image a separate 
marker having no unique geometric characteristics, by utilizing the a two-dimensional or 
three-dimensional positional relationship of the first marker, and updates the three- 
dimensional position and orientation parameter of the object with respect to the image 
acquisition apparatus, by utilizing the three-positional position of the marker with respect to 
>P)^ V the image acquisition apparatus. 

JM, (Original) A three-dimensional position and orientation sensing apparatus 
according to claiin^v, wherein the image acquisition apparatus acquires a plurality of images 
and utilizes the plurality of images. 

A\ . (Original) A three-dimensional position and orientation sensing apparatus 
according to claim,2^wherein markers are mounted on the plane of a sensor probe, and the 
sensor probe is used as an object to be measured, whereby to estimate the relative position and 
orientation of the sensor probe and to measure the position of a probe tip of the sensor probe. 



^^2^Cancelled 




J. (New) A three-dimensional position and orientation sensing apparatus 

compnsmg: 
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an image input section which inputs an image acquired by an image acquisition 
apparatus and showing at least three markers having color or geometric characteristics as one 
image, three-dimensional positional information of the markers with respect to an object to be 
measured being known in advance; 

a region extracting section which extracts a region corresponding to each 
marker in the image; 

a marker identifying section which identifies the individual markers based on 
the color or geometric characteristics of the markers in the extracted regions; and 

a position and orientation calculating section which calculates the three- 
dimensional position and orientation of the object to be measured with respect to the image 
\^ acquisition apparatus, by using positions of the identified markers in the image input to the 



image input section, and the three-dimensional positional information of the markers with 



respect to the object to be measured. 

(New) A three-dimensional position and orientation sensing apparatus 



comprising: 

an image input section which inputs an image acquired by an image acquisition 
apparatus and showing at least foxir markers having color or geometric characteristics as one 
image, three-dimensional positional information of the markers with respect to an object to be 
measured being known in advance; 

a region extracting section which extracts an area corresponding to each 
marker in the image; 

a marker identifying section which identifies the individual markers based on 
the color or geometric characteristics of the markers in the extracted regions; 
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a marker selecting section which selects three markers from the markers 
identified by the marker identifying section; 

a parameter calculating section which calculates a plurality of sets of 
parameters for calculating the three-dimensional position and orientation of the object to be 
measured with respect to the image acquisition apparatus, by utilizing positions of the three 
markers in the image input to the image input section which are selected by the marker 
selecting section and three-dimensional positional information of each marker with respect to 
the object to be measured; and 



application of the parameters calculated by the parameter calculating section, to markers not 
selected by the marker selecting section. 



apparatus and showing at least three markers having color or geometric characteristic as one 
image, three-dimensional positional information of the markers with respect to an object to be 
measured being known in advance; 

a region extracting section which extracts a region corresponding to each 
marker in the image; 

a marker identifying section which identifies the individual markers based on 
the color or geometric characteristics of the markers in the extracted regions; 



a parameter selecting section which selects one set of parameters by evaluating 




(New) A three-dimensional position and orientation sensing apparatus 



compnsing: 



a image input section which inputs an image acquired by an image acquisition 
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a distance estimating section which estimates a distance from the image 
acquisition apparatus to each marker based on a size of the extracted region in the image 
including the identified marker; and 

a position and orientation calculating section which calculates the three- 
dimensional position and orientation of the object with respect to the image acquisition 
apparatus, by using the estimated distance to each marker, a position of the marker in the 
image and three-dimensional positional information of this marker with respect to the object. 



(New) A three-dimensional position and orientation sensing apparatus 



compnsmg: 



an image input section which inputs an image acquired by an image acquisition 



apparatus and having marker^three-dimensional positional information of which with respect 
to an object to be measured is known in advance; 

a region extracting section which reduces the size of the input image and 
extracting a region corresponding to each marker in the reduced image; and 

a position and orientation calculating section which calculates the three- 
dimensional position and orientation of the object with respect to the image acquisition 
apparatus, by using the extracted position of each marker in the image and three-dimensional 
positional information of each marker with respect to the object. 



(New) An apparatus for measuring a relative three-dimensional position 
and orientation of an object with respect to an image acquisition apparatus by measuring 
image positions of at least three markers mounted on the object or near the object, based on an 
image showing the markers, acquired by the image acquisition apparatus, wherein the markers 



15 
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c^^ vnrc^^s 
have color or geometric characteristicsand three-dimensionaTpositions of the markers are 

known in advance, and the measuring apparatus comprising: 

a section which extracts a region corresponding to each of the markers from 
within the image of the markers; 

a section which identifies the individual markers by calculating color or 
geometric characteristics of the region corresponding to each of the markers within the image; 
and 

a section which estimates the relative three-dimensional position and 
orientation of the object with respect to the image acquisition apparatus, based on the three- 
dimensional positions of the markers. ^ 
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